Visual communication using a robotic device

ABSTRACT

Technology is described for visually communicating using a robotic device. An example of a method can include a video feed sent from the video camera of the robotic device to the remote user. A projection surface identified in the video feed can then be sent to the remote user using an application. Another operation can be obtaining an image from the remote user using the application. The image created by the remote user can then be projected on the projection surface.

BACKGROUND

Robotic devices can provide services for humans. Examples of a usefulrobotic device can be a simple autonomous robot to provide services toan elderly person or to patrol a workplace at night. Robotic devices canalso have applications that control the operations that are performed bythe robotic device. For example, one application may include functionsfor accomplishing navigation tasks by localizing or estimating thecurrent location of the robotic device and for navigating reliably toreach locations in the environment. Other example applications caninclude telecommunications, photo capture, video playback, audioplayback, navigation, and video conferencing abilities.

An example of a video conference application can use a robotic devicewhere a video screen on a robotic device displays a video image of aremote user to communicate with a local user at the location where therobotic device is located. The video screen can be viewed by the localuser at the robotic device's location and the remote user can view thelocal user using a camera mounted in the robotic device. This allows atwo-way video conversation to take place using audio and video. Ifdesired, the robotic device may follow the local user around in abuilding during the video conversation. However, the interaction betweenthe local user and the remote user is generally limited to the screen ofthe robotic device and what the camera captures of the local user.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. While certaindisadvantages of prior technologies are noted above, the claimed subjectmatter is not to be limited to implementations that solve any or all ofthe noted disadvantages of the prior technologies.

Various examples are described for visually communicating using arobotic device. An example of a method can include a video feed sentfrom the video camera of the robotic device to the remote user. Aprojection surface identified in the video feed can also be sent to theremote user using an application. Another operation can be obtaining animage from the remote user using the application. The image from theremote user can then be projected on the projection surface.

An example of a system for projecting a remotely created image on aprojection surface using a robotic device can also be provided. Thesystem can include a robotic device configured to navigate to a locationwith the projection surface. A video camera can be included with therobotic device to capture a point of view with the projection surface.In addition, a telepresence application can display a video stream fromthe video camera to a remote user and receive an image from the remoteuser. For example, the remote user can draw an image on the videostream. A projector for the robotic device in communication with thetelepresence application can project the image received from the remoteuser onto the projection surface.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a system forprojecting a remotely created image onto a projection surface using arobotic device.

FIG. 2 is an example of a flowchart illustrating a method for visuallycommunicating using a robotic device.

FIG. 3 is a flowchart illustrating an example of a method for creatingand projecting a remotely created image on a projection surface using arobotic device.

FIG. 4 is a block diagram illustrating an example of a detailed methodfor visually communicating using a robotic device.

DETAILED DESCRIPTION

Reference will now be made to the examples illustrated in the drawings,and specific language will be used herein to describe the same. It willnevertheless be understood that no limitation of the scope of thetechnology is thereby intended. Alterations and further modifications ofthe features illustrated herein, and additional applications of theexamples as illustrated herein, which would occur to one skilled in therelevant art and having possession of this disclosure, are to beconsidered within the scope of the description.

As part of a robotic device's telepresence session, a remote user can beprovided with technology to virtually draw in a robotic device'senvironment in order to convey ideas, provide sketches, play games, orotherwise interact with a local user. The term virtually drawing canmean using light from a projector of a robotic device to draw figures,shapes and produce other indicia on a surface local to the roboticdevice. A telepresence session can refer to using technology to enable aremote user to feel as if the remote user is present or to give theremote user the appearance that the remote user is present at a locationother than the remote user's true location. This technology can enable aremote user to draw on or project images, pictures, or shapes onspecific areas, objects, furniture, animals, or people during a robotictelepresence session by leveraging robotic mobility, image projectorsfor a robotic device, remote navigation, and drawing applications.

As part of a communication session, the remote user can run anapplication that accesses the robotic device's video feed. Thatapplication may enable the remote user to view or select an area in therobotic device's environment and create an image (e.g., drawing) thatcan be mapped directly onto the video feed of the robotic device. Forexample, the creation of an image can include drawing electronic lines,figures, and indicia. The user's drawing motion (i.e., cursor position)on the video feed can be translated into images, drawings, or objectsthat are projected through the robotic device's projection system as thedrawing motions are performed or immediately thereafter. Local andremote users are able to view the images or drawings that are beingprojected on the projection area as changes are made by the remote user.This continuous update and feedback can give the remote user a feelingthat the remote user is drawing directly onto a remote projectionsurface using light from the robotic device's projector.

FIG. 1 illustrates an example system for projecting a remotely createdimage on a surface using a robotic device. The system can include therobotic device 102 configured to navigate to a location with aprojection surface 104. The robotic device can be a mobile robot withwheels or legs, a stationary robot with moveable limbs, a robotic arm,or another type of robotic device. A video camera 110 can be includedfor the robotic device to capture a point of view with the projectionsurface. To capture a desired point of view, the robotic device may moveto a desired location and/or orient the video camera to capture a pointof view.

A telepresence application 120 on the robotic device can be used to sendand display a video feed 130 from the video camera 110 to a remote user150. A remote user can be a user located on a network different than therobotic device's network, and the two networks can communicate through aconnecting LAN (Local Area Network), WAN (Wide Area Network) or theInternet. The robotic device's video feed sent to the remote user canshow what is being captured by the robotic device's camera as the videofeed is streamed to the remote user.

The remote user 150 can access the robotic device 102 through the remotetelepresence application 140 on a personal computer, mobile phone,computing tablet, a game console, or another computing device. Theremote application can be a stand-alone application or embedded in abrowser. The remote module (e.g., client module) of the telepresenceapplication can be used to display the video stream to the remote user.As a more specific example, the overall telepresence application can becomposed of a robotic application component and a remote clientcomponent which communicate with each other.

A projector 112 can be included for the robotic device and the projectorcan be in communication with the telepresence application 140 to projectan image received from the remote user on the projection surface 104.The projector can be mounted so that the projection surface can bechanged using motors or other robotic actuator mechanisms.

The remote user 150 can create an image 132 by drawing or embeddingobjects and figures using the remote telepresence application (e.g.,remote client component). For example, the remote user can virtuallydraw on the video feed 130 displaying the projection surface in thevideo feed, and project virtual drawings obtained from the remote useronto the projection surface. The image with drawn objects or embeddedobjects can be sent back to the telepresence application on the roboticdevice, and the video feed can also show the robotic device'spoint-of-view as combined with the remote user's drawings or images. Inanother example, the remote user may draw on a separate drawing area inthe telepresence application and then the drawings or images can beoverlaid onto the video stream when the user activates a user interfacecontrol (e.g., a graphical button).

A local user 106 can also interact with objects drawn in an image on theprojection surface 104. The local user can be located in the samephysical environment as the robotic device during a robotic telepresencesession. The video camera 110 of the robotic device can detect aposition of the local user to enable the user to interact with an objectdrawn in the image in the projection. For example, the position of thelocal user with respect to the projection surface can be detected toallow the local user to virtually move, hit, deform, or change drawingelements. A further example of a robotic telepresence session can be acommunication session in which a remote user takes over control of arobotic device and receives video, audio and sensory information fromthat robotic device and the local user can interact with figures thatthe remote user is drawing while the figures are drawn. The local useris illustrated in FIG. 1 with the projection surface between the localuser and the robotic device 102. However, the local user can be in anynumber of other positions with respect to the projection surface and theprojector. For example, the local user may be between the projectionsurface and the robotic device, behind the robotic device, or beside therobotic device. These varying local user positions can allow the localuser to introduce a hand, arm or other pointer into the projection area.

A depth camera 110 can be used in addition to the visual camera toidentify a depth and shape of a projection surface. The drawing or imageprojected onto the projection surface can also be conformed to the shapeof the projection surface as detected by the depth camera. For example,a drawing can have a graphical transformation applied such as a warpingfunction or another graphical transformation so that the drawn imagedoes not appear significantly distorted in the projection but may appearcloser to what the remote user had originally drawn without thedistortion introduced by the projection onto a physical object.

The robotic device 102 can further include a computing device 180 with ahardware processor device 182, a hardware memory device 184, a localcommunication bus 186 to enable communication between hardware devicesand components, and a networking device 188 for communication across anetwork with the compute nodes, processes on the compute nodes, orcomputing devices on other agents. The computing device can be used toexecute the telepresence application and process other robotic devicefunctions. The robotic device 102 can also include a display screen 122to display the robotic device's status, application statuses, networkcommunications, or other robotic device information. A navigation module124 can also provide the capability for the robotic device to navigatethrough an environment autonomously or under the remote user'sinstructions.

FIG. 2 illustrates an example of a method for communicating using arobotic device. A video capture session for a remote user of a roboticdevice can be initiated. This allows the remote user to view where therobotic device is currently located and the remote user, local user, oranother user can instruct the robotic device to navigate to a locationin the environment. The method can include the operation of instructinga robotic device to set a view point of a video camera based on controlinput from a remote user, as in block 210. This can include moving therobotic device to a desired location and/or setting the orientation of acamera carried by the robotic device. Alternatively, the control of therobotic device may be autonomous and controlled by a softwareapplication with data.

A video feed can be sent from the video camera of the robotic device tothe remote user, as in block 220. A projection surface that has beenidentified in the video feed can also be displayed to the remote userusing an application, as in block 230. The application may be atelepresence application where a user is able to control and send datato a robotic device with a robotic application component and a remoteclient component. In one example, the telepresence application candetect a projection area in the video feed that is believed to be a goodfit for projecting onto. The remote user may also have the ability toadjust the automatically selected projection area by activating anadjustment mode and moving around graphical handles on a boundedprojection area. Alternatively, the remote user can select the area inthe video stream that is desired as the projection area by tracing adesired area.

An image can then be obtained from the remote user using theapplication, as in block 240. The image can be obtained by enabling theremote user to draw on the projection surface identified in the videofeed. In addition, the remote user may provide or insert graphicalobjects that can be displayed on the projection surface.

The remote user's image or drawings can then be projected on the displaysurface, as in block 250. The projector for the robotic device can bealigned with the view point of the camera and this can allow the remoteuser to virtually write on the projection surface that the remote useris viewing. The image can include the remote user's free hand drawings,typed text, embedded photos or other image material created by theremote user. Thus, the drawn images can appear to be on the selecteddisplay surface or projection area due to the projected drawing.

In one example, once the remote user has aligned a drawing area with theprojection surface, the camera may be panned and tilted relative to theprojector to view other areas in the remote scene. Then the telepresenceapplication or robotic device can dynamically adjust the drawing area toremain at a fixed location and orientation with respect to the remoteenvironment. To the remote user, such dynamic drawing area adjustmentscan cause a portion of the drawing surface to appear to move off theedges of the field of view as the camera moved, and the area availablefor drawing may be much less than the original drawing area. In otherwords, the drawing area may appear fixed with respect to the selectedprojection surface. With a sufficiently large pan or tilt motion, theentire drawing area may move off the field of view based on the currentcamera position. However, when the camera is adjusted back to theoriginal position, the drawing area can return to the screen and be bothvisible and active for additional drawing. This function can allow theuser to draw in one area of the remote environment as defined by thecamera orientation but still have the freedom to control the camera tolook at areas other than the area which maps to the projector's field ofview. The remote user's display can be dynamically adjusted toaccurately reflect any changes of alignment between the camera andprojector and the associated change to the available drawing area.

Further, a local user may interact with objects drawn in the image onthe projection surface. The camera of the robotic device can detect aposition of the local user to enable the local user to interact withobjects drawn in the image on the projection surface. A depth camera canalso be used to allow the local user to interact with objects drawn bythe remote user. For example, a depth camera can be used to helpidentify the local user's index finger and the local user can use theirindex finger to draw on the projection area too.

The depth camera can also be used to identify a shape of a projectionsurface using a depth camera of the robotic device. Knowing the shape ofthe projection area can allow the application to modify the image ordrawing projected onto the projection surface to accommodate a shape ofthe projection surface. This allows the image to be viewed with lessdistortion on a variety of projection surfaces. For example, if theremote user draws some lettering that is projected on a surface at anangle to the robotic device, then the lettering can be graphicallytransformed so that the lettering is more readable on the angledprojection surface.

FIG. 3 further illustrates an example of the technology to enable a userto draw virtually in a remote environment during a robotic telepresencesession. This can be called light ink for a robotic device. Initially, atelepresence session can be started by a remote user or a local user, asin block 302. A robotic telepresence session may entail a remote userrunning an application on a personal computer, mobile device, or gameconsole to access a robotic device's video, audio and sensory data withthe ability to navigate the robotic device.

The robotic device can then send point of view video feed to the remoteuser, as in block 304. The robotic device can be guided by a user to aspecific location and point-of-view in the robotic device's environment,as in block 306. After the robotic device receives the instructions,then the robotic device can move to the location and adjust apoint-of-view of the robotic device and the camera to match the remoteuser's selections and/or directions, as in block 308. At this point, theremote user can switch to the drawing mode (e.g., light ink mode), as inblock 310.

This technology can leverage the mobility of a robotic device to allow aremote user to have the robotic device navigate to and select a visiblesurface in a robotic device's environment as a drawing surface. Thisnavigation and selection can be performed via the telepresenceapplication. The selected location can be any item or surface visiblethrough the robotic device's point-of-view that can be reached by theprojector. The projector located within the robotic device can projectdrawings, sketches, shapes or pictures that the remote user wishes tooptically overlay on the robotic device's environment and share with alocal user. This allows the remote user to draw remotely and virtuallyon surfaces local to the robotic device.

A remote user can position the robotic device to face and project on asurface. The positioning functionality can be part of the roboticdevice's navigation system. By positioning the robotic device correctly,the user can project on a viewable surface. The robotic device'sprojector can be aligned with the robotic device's point-of-view sent tothe remote user which may make the selection of the projection area orprojection surface easier, as in block 312. The robotic device canfurther identify a projection area and send the selection coordinates toa remote user for viewing, as in block 314. By aligning the projectorwith the robotic device's camera point-of-view, the remote user is ableto use the projector to draw on surfaces seen in that camerapoint-of-view. In addition, a projection surface can be shown on therobotic device's video feed, as in block 316. This compositing mayinclude overlaying the location of the candidate projection surface onthe robotic device's video feed shared with the remote user.

Once a projection surface has been selected, the remote user can use therobotic telepresence application to draw on that projection surface.Previously existing telepresence applications do not enable the user todraw directly on a remote video feed. Instead such previous applicationshave provided a local whiteboard that is replicated over to a remotescreen typically located on a personal computer (PC) or televisionmonitor. In this technology, the whiteboard is the environment as seenby the robotic device and the user can project drawings directly ontothat environment.

A remote user can select the type of drawing tool that the remote userwould like to use, or a default selection can be used, as in block 320.The user can select between preset shapes such as star shaped stamps,tic-tac-toe boards, paint brush shape, color, size, style, photos, orother pictures to add in the drawing tool.

Shapes and/or objects can be inserted or applied to the projectionsurface that are dynamic or animated. Examples include a spinning star,pulsating light sphere, or bouncing ball. More complicated and dynamicanimations can be added such as a butterfly with flapping wings, whichmay move around the projection surface to make the butterfly appear morelife-like. These shapes or animations can be pre-programmed with certainbehaviors so that the remote user can just add the shapes or animationsto the projection surface and the user does not have to manually movethem about. In addition the remote user can customize the shapes oranimations in some way (e.g. by modifying the type of butterfly, etc.).In another example, the user may desire to display engineering drawingswith perspective, CAD (computer-aided design) drawings, or other 3D(three dimensional) modeled objects that can be displayed on aprojection surface.

Once a shape or animation has been selected, the remote user caninteract with the remote video feed either using touch input, gestureinput, pointer input, or a mouse type input. As the user locates (e.g.,hovers) the cursor over the video feed, the cursor can change to reflectthe selected shape or brush style and size, as in block 322. As thecursor moves around the video feed, the robotic device's projector canproject the cursor within the robotic device's environment in relationto the projection surface, as in block 324. The cursor shape seen by theremote user can be a projection of the cursor in the robotic device'senvironment and can be seen via the robotic device's video feed.

Using the equivalent input signal of a mouse down and/or drag movement,the remote user can draw on the remote video feed or drop a shape on thevideo feed, as in block 326. As the drawing occurs, the projector canshow the shape or user's drawing projected in the robotic device'senvironment, as in block 328. In addition, the user can clear the areato create new drawings or drop new shapes. The user can also save thecombination of shapes and drawings created to reuse for projection on adifferent surface.

By standing in front of the camera, the local user can interact withdrawings, shapes, or objects by moving various parts of the local user'sbody (e.g. arms, head, feet, legs). The cameras can interpret usermovement and the robotic device can correlate the local user's movementwith the projected shapes or objects such that a change in the projectedimage occurs. The local user may be located between the projector andthe projector surface or the local user may be located next to theprojection surface to interact with objects in the projected image usinggestures to produce a change in the projected image. This change canthen communicated back to the remote telepresence application and can beshown to the remote user. Such feedback can allow the remote user andthe local user to view the change similarly. Interactive shapes orobjects can be predefined to allow the local and remote user to interactwith the whole object or with parts of the object.

The remote telepresence application can also assist the remote user inwhat the remote user may be drawing based on information provided aboutthe projection surface or projection surface coordinates from the camerausing either depth data or using edge detection on the video stream. Asan example, if a contiguous bordered area is found (e.g. a picture frameor cupboard door) in a camera image, the remote user can utilize anauto-fill function to automatically project a different color that isbounded by this area, without having to create a rectangle. The remoteuser can also be assisted when drawing shapes or objects so that theshapes or objects “snap” to existing fixtures or features of theprojection surface.

FIG. 4 illustrates an additional example of a method for communicatingusing a robotic device. The method can include sending a video feed froma video camera of the robotic device to a remote user, as in block 410.A robotic device can be instructed to move to a location and set a viewpoint of a video camera based on control input from the remote user, asin block 420. The control input from the remote user can includerelative local coordinates, location tags, absolute global positioningcoordinates, or joystick type of controls.

The projection surface coordinates can be identified in the video feedand displayed to the remote user, as in block 430. The projectionsurface coordinates can be displayed to the remote user in the videofeed using a telepresence application, as in block 440.

A drawing by the remote user can be captured at the projection surfacecoordinates on the video feed, as in block 450. The drawing from theremote user can be submitted to the robotic device using thetelepresence application, as in block 460. Then the remote user'sdrawing can be projected to the projection surface using a projector ofthe robotic device, as in block 470.

The technology described includes the ability for a remote user tovirtually draw on surfaces in a robotic device's environment using arobotic device's projector. A remote user's drawing can be convertedinto light patterns as the drawing is created and those patterns can beprojected onto the robotic device's environment using the roboticdevice's projector.

Some of the functional units described in this specification have beenlabeled as modules, in order to more particularly emphasize theirimplementation independence. For example, a module may be implemented asa hardware circuit comprising custom VLSI circuits or gate arrays,off-the-shelf semiconductors such as logic chips, transistors, or otherdiscrete components. A module may also be implemented in programmablehardware devices such as field programmable gate arrays, programmablearray logic, programmable logic devices or the like.

Modules may also be implemented in software for execution by varioustypes of processors. An identified module of executable code may, forinstance, comprise one or more blocks of computer instructions, whichmay be organized as an object, procedure, or function. Nevertheless, theexecutables of an identified module need not be physically locatedtogether, but may comprise disparate instructions stored in differentlocations which comprise the module and achieve the stated purpose forthe module when joined logically together.

Indeed, a module of executable code may be a single instruction, or manyinstructions, and may even be distributed over several different codesegments, among different programs, and across several memory devices.Similarly, operational data may be identified and illustrated hereinwithin modules, and may be embodied in any suitable form and organizedwithin any suitable type of data structure. The operational data may becollected as a single data set, or may be distributed over differentlocations including over different storage devices. The modules may bepassive or active, including agents operable to perform desiredfunctions.

The technology described here can also be stored on a computer readablestorage medium that includes volatile and non-volatile, removable andnon-removable media implemented with any technology for the storage ofinformation such as computer readable instructions, data structures,program modules, or other data. Computer readable storage media include,but is not limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tapes, magnetic disk storage orother magnetic storage devices, or any other computer storage mediumwhich can be used to store the desired information and describedtechnology.

The devices described herein may also contain communication connectionsor networking apparatus and networking connections that allow thedevices to communicate with other devices. Communication connections arean example of communication media. Communication media typicallyembodies computer readable instructions, data structures, programmodules and other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia. A “modulated data signal” means a signal that has one or more ofits characteristics set or changed in such a manner as to encodeinformation in the signal. By way of example, and not limitation,communication media includes wired media such as a wired network ordirect-wired connection, and wireless media such as acoustic, radiofrequency, infrared, and other wireless media. The term computerreadable media as used herein includes communication media.

Furthermore, the described features, structures, or characteristics maybe combined in any suitable manner in one or more examples. In thepreceding description, numerous specific details were provided, such asexamples of various configurations to provide a thorough understandingof examples of the described technology. One skilled in the relevant artwill recognize, however, that the technology can be practiced withoutone or more of the specific details, or with other methods, components,devices, etc. In other instances, well-known structures or operationsare not shown or described in detail to avoid obscuring aspects of thetechnology.

Although the subject matter has been described in language specific tostructural features and/or operations, it is to be understood that thesubject matter defined in the appended claims is not necessarily limitedto the specific features and operations described above. Rather, thespecific features and acts described above are disclosed as exampleforms of implementing the claims. Numerous modifications and alternativearrangements can be devised without departing from the spirit and scopeof the described technology.

1. A method for visually communicating using a robotic device,comprising: sending a video feed from the video camera of the roboticdevice to a remote user; displaying a projection surface identified inthe video feed to the remote user using an application; obtaining animage from the remote user using the application; and projecting theremote user's image on the projection surface.
 2. The method as in claim1, further comprising: enabling the remote user to draw on theprojection surface identified in the video feed; and projecting drawingsfrom the remote user to the projection surface using a projector withthe robotic device.
 3. The method as in claim 1, wherein the applicationis a telepresence application with a robotic application component and aremote client component.
 4. The method as in claim 1, further comprisingaligning a projector for the robotic device with a view point of a videocamera.
 5. The method as in claim 1, further comprising initiating avideo capture session for a remote user of a robotic device.
 6. Themethod as in claim 1, further comprising instructing the robotic deviceto set a view point of a video camera based on control input from aremote user.
 7. The method as in claim 1, further comprising enabling alocal user to interact with objects drawn in the image on the projectionsurface.
 8. The method of claim 1, further comprising: identifying ashape of the projection surface using a depth camera of the roboticdevice; and; modifying the image projected onto the projection surfaceto accommodate the shape of the projection surface.
 9. The method as inclaim 1 further comprising enabling a drawing area to appear fixed withrespect to the projection surface while changing a camera view.
 10. Asystem for projecting a remotely created image on a projection surface,the system comprising: a robotic device configured to navigate to alocation with the projection surface; a video camera for the roboticdevice to capture a point of view with the projection surface; atelepresence application to display a video stream from the video camerato a remote user and receive an image from the remote user; and aprojector for the robotic device in communication with the telepresenceapplication to project the image received from the remote user on theprojection surface.
 11. The system as in claim 10, wherein thetelepresence application further comprises with a robotic applicationcomponent and a remote client component.
 12. The system as in claim 10,wherein the remote client component allows the remote user to draw onthe video feed containing the projection surface from the video cameraand project drawings obtained from the remote user on the projectionsurface.
 13. The system as in claim 10, wherein a local user is able tointeract with objects drawn in the image on the projection surface. 14.The system as in claim 10, wherein the camera of the robotic device candetect a position of a local user to enable the user to interact with anobject drawn in the image on the projection surface.
 15. The method ofclaim 10, further comprising a depth camera to identify a shape of aprojection surface, wherein the image projected onto the projectionsurface is conformed to the shape of the projection surface as detectedby the depth camera.
 16. A method for visually communicating using arobotic device, comprising: sending a video feed from a video camera ofthe robotic device to a remote user; instructing a robotic device tomove to a location and set a view point of the video camera based oncontrol input from the remote user; identifying projection surfacecoordinates in the video feed displayed to the remote user; displayingthe projection surface coordinates to the remote user in the video feedusing a telepresence application; capturing a drawing on the video feedat the projection surface coordinates by the remote user; submitting thedrawing from the remote user to the robotic device using thetelepresence application; and projecting the remote user's drawing tothe projection surface coordinates using a projector of the roboticdevice.
 17. The method as in claim 16, further comprising: enabling alocal user, located between the projector and projection surfacecoordinates, to interact with objects in the remote user's drawing usinggestures to effect a change in the drawing; and communicating the changeback to the remote user so that the remote user and local user view thechange similarly.
 18. The method as in claim 16, further comprisingenabling the telepresence application on the robotic device to selectthe projection surface for the projector to project the drawing on. 19.The method as in claim 16, further comprising projecting the drawing tothe projection surface to match the shape of the projection surface. 20.The method as in claim 16, further comprising assisting the remote userin drawing based on depth data aspects or using edge detection aspectsof the projection surface determined by the telepresence applicationusing the camera.